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1 Introduction 

In her speech to the Conservative party conference on 5 th October 
2016, Prime Minister Theresa May articulated her vision for a 
“country that works for everyone” that takes its place on the new, 
post-Brexit world stage: 

“I want us to be a country where it doesn’t matter where you were 
born, who your parents are, where you went to school ...All that 
should matter is the talent you have and how hard you’re prepared 
to work.” 

The release of new data from the Programme for International 
Student Assessment (PISA) alongside the Trends in International 
Mathematics and Science Study (TIMSS) provides a unique 
opportunity to examine the impact of disadvantage on pupils in 
England from an international perspective. 

In this report, we focus on performance in mathematics to answer 
three questions: 

• How well is England doing at supporting the 
mathematics performance of disadvantaged pupils 
compared with other countries, and has this changed 
over time? 

• Are there aspects of mathematics where disadvantaged 
pupils are weaker and support could be targeted? 

• In what circumstances do disadvantaged pupils tend to 
beat the odds to perform better, and are there lessons 
that can be applied more widely? 


2 At a glance 

The impact of socio-economic background on mathematics 
performance in England can be seen from the most to least 
disadvantaged. As socio-economic background of pupils increases, 
so does average mathematics performance; the gap between the 
most and least disadvantaged is equivalent to over three years’ of 
schooling. 

However, many factors other than socio-economic background also 
affect performance, and these other factors are relatively more 
important to pupil performance in England than in other countries. 
Furthermore, the impact of socio-economic background is lower 
among the most disadvantaged half of the population. 

Examining other countries and the lack of change over time 
suggests that it is very difficult for countries to reduce the impact of 
socio-economic background on performance. 

Disadvantaged pupils who perform better than average, given their 
socio-economic background, tend to be autumn-born, are more 
confident in their abilities, and are less likely to truant. 

Our analysis shows that pupils would benefit from a more granular 
measure of deprivation; that summer-born pupils need a strategy to 
ensure they are not left behind; that further research would be 
beneficial to understand better the characteristics of pupils that 
perform better than their background would predict; and that 
schools should be further supported in tackling underperformance 
of disadvantaged pupils. 
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3 Background 

Disadvantaged pupils 

The Government’s current focus on families that are “just about 
managing” follows a number of initiatives introduced since 2010 that 
target disadvantaged pupils. 

Since 2011, disadvantaged pupils have benefited from the 
pupil premium. 

In 2011, the Department for Education the pupil premium was 
introduced as a new funding stream for schools that was to be used 
to improve outcomes for disadvantaged pupils. A number of 
different groups were eligible, including pupils from low income 
families entitled to free school meals (FSM), and pupils who are or 
have been in the care system. The value of this funding has 
increased overtime, and in 2016/17 each primary school FSM pupil 
attracted an additional £1 ,320 to their school for example. 

The Education Endowment Foundation (EEF) was set up to 
provide evidence for schools of what works to raise the 
attainment of disadvantaged young people. 

EEF is an independent charity established by the Government in 
201 1 , to accompany the introduction of the pupil premium. It funds 
research into a wide range of classroom practices and 
interventions. It also communicates the evidence to schools through 
its online toolkit and scale-up campaigns, in order to equip schools 
to make well-informed decisions about the approaches likely to be 
most effective in supporting disadvantaged young people. 


The Social Mobility Commission and Fair Education Alliance 
have further emphasised the need to ‘close the gap’. 

The Social Mobility Commission is an independent statutory body 
established to promote, monitor and research social mobility in 
England. In their 201 6 report State of the Nation 2016: Social 
Mobility in Great Britain, the Commission acknowledged some 
success in narrowing the attainment gap between disadvantaged 
pupils and their peers. Flowever it also highlights that there is still a 
lot that needs to be done with our education system to minimise the 
gap. 

The Fair Education Alliance was established in 2014, a collection of 
teachers’ organisations, children’s charities and employers setting 
targets to narrow the gap in literacy and mathematics between 
advantaged and disadvantaged pupils in England. It also aims to 
develop pupil’s resilience and wellbeing, to benefit them in further 
education. 

The current Government is introducing further initiatives. 

In the post-Brexit world our young people may face additional 
uncertainties as they complete their education and enter the world 
of work. Recognising the need to improve the opportunities for 
disadvantaged pupils, the Department for Education recently 
released details of a social mobility package (Department for 
Education & Greening, 2016) with the aim to provide funding to 
improve the social mobility of children in six ‘opportunity areas’ in 
England and includes a teaching and leadership innovation fund. 

In December 2016, the Government consultation on ‘Schools that 
work for everyone’ closed. This consultation proposed expansion of 
independent schools, higher education institutions, selective and 
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faith schools, with the aim to increase the number of ‘good’ school 
places. Expansion is particularly necessary at secondary school 
due to a projected increase in pupil numbers by around 10% 
between 2016 and 2020 (Department for Education, 2016). 

The importance of mathematics 

The numeracy skills of workers in England are a bigger 
predictor of economic returns than in many countries. 

Attainment in mathematics at school has life-long consequences. A 
high level of cognitive skill, including basic numeracy skills, is highly 
regarded in the UK labour market, and provides higher economic 
returns than many other countries (Crawford et al, 2011). As part of 
the Government’s reforms to raise standards in mathematics 
education, and prompted by England’s unchanged performance in 
PISA 2012 compared with previous surveys, Minister Truss 
announced a network of ‘maths hubs’ to bring England in line with 
top-performing east Asian Countries (Japan, Singapore and China). 
This scheme, funded by the Department for Education, set 32 
schools and academy trusts as lead hubs across England as 
models implementing an Asian-style mastery approach to 
mathematics. 

Disadvantaged pupils continue to achieve lower results in 
mathematics. 

Since the introduction of the pupil premium, some progress has 
been made: the attainment gap for secondary pupils achieving five 
or more GCSEs at A*-C grades including English and mathematics 
has closed by 1.6 percentage points (GB. Parliament. HoC. 
Committee of Public Accounts, 2015). It closed slightly more in 


schools participating in the EBacc than those who were not, and 
researchers have argued more is needed to ensure that 
disadvantaged pupils have fair access to these subjects (Allen et 
al., 2016). 

There is not yet key stage 4 data available for the first pupils to 
have benefited from the pupil premium for the whole of their 
schooling from key stage 2 onwards, and which therefore allows 
their progress to be tracked. However, data from pupils who sat 
their GCSEs in 2014 shows that pupils entitled to free school meals 
continued to make less progress compared with their peers: 45.6% 
of FSM pupils made the expected level of progress in mathematics 
compared with 68.8% of all other pupils, a gap of 23.2 percentage 
points. (Department for Education, 2015). 

What can we learn from international 
surveys? 

The release of the Programme for International Student 
Assessment (PISA) 2015 data on 6 December 2016 alongside the 
Trends in International Mathematics and Science Study (TIMSS) 
2015 on 29 November 2016 provides rich data to look at the 
mathematics performance of pupils in England over time, and to 
make comparisons with other countries. 

Both PISA and TIMSS assess mathematics (amongst other 
subjects). PISA assesses mathematical literacy and aims to 
understand how well pupils are able to apply their mathematical 
skills to unfamiliar and real-world contexts. TIMSS has a greater 
emphasis on assessing knowledge and is more curriculum 
focussed. Both surveys collect contextual information from pupils 
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and teachers so that we can analyse to what extent pupil 
characteristics and their experiences in school impact on 
performance. 

International surveys provide a unique opportunity to examine 
the impact of disadvantage on pupils in England. 

The pupil questionnaire responses in PISA and TIMSS provide 
evidence about disadvantage. In particular, PISA’s Economic, 

Social and Cultural Status (ESCS) measure provides a more 
sophisticated measure of disadvantage than the binary ‘FSM or not’ 
measure traditionally considered (see Chapter 4). The studies also 
enable us to consider the issue from an international perspective, 
comparing England’s success in closing the gap to that of other 
countries. 

Previously, data from the surveys has shown that although socio- 
economic status does not wholly determine a pupil's academic 
performance, it does have an important influence (OECD, 2014). In 
PISA 2012, a socio-economically disadvantaged background was 
identified as an important risk factor for the 1 3 million 1 5 year old 
pupils who were low performers in at least one subject (OECD, 
2016b). 

The PISA 2015 generation 

Pupils participating in PISA 2015 in England were born at the turn 
of the millennium, and over the course of their schooling have 
experienced a number of reforms that may have affected their 
outcomes. Figure 2.1 highlights a selection of these policies which 
could have had an impact on the results reported in this analysis. 


Figure 2.1 Policy timeline for the PISA 201 5 generation 


Year 

Cohort 

age 

Events 

1999 

- 

Sure Start children’s centres introduced 

2000 

0 

First academies announced 

General Teaching Council established 

2001 

1 

Labour Government re-elected 

2002 

2 

Teach First launches in London 

2003 

3 

London Challenge launched, with a particular 
focus on disadvantage 

2004 

4 

Every Child Matters becomes law, focused on five 
key outcomes for children 

2005 

5 

Labour Government re-elected 

2006 

6 

Teach First begins expansion, firstly to 

Manchester 

2007 

7 


2008 

8 

Abolition of Key Stage 3 testing announced, to 
take effect from 2009 

2009 

9 

Abolition of Key Stage 2 science testing 
announced, to take effect from 2010 

2010 

10 

Conservative-Lib Dem coalition Government 

English Baccalaureate announced 

Academisation accelerated 

2011 

11 

Pupil premium introduced and EEF established 

New Ofsted framework focuses on achievement 

2012 

12 

General Teaching Council abolished 

Schools Direct route into teaching introduced 

2013 

13 


2014 

14 


2015 

15 

PISA 2015 testing takes place 
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PISA 2015 pupils have experienced a secondary education 
with greater emphasis on traditional subjects. 

In January 201 1 (when this cohort were in year 6), the English 
Baccalaureate (EBacc) was introduced as a new school 
performance measure. A pupil achieves the EBacc by studying 
English, mathematics, two sciences, history or geography and a 
foreign language at GCSE. 

New school performance measures (Progress 8 and Attainment 8) 
were also announced in 2013, to be applied from summer 2016 and 
therefore being used for the first time with the cohort that sat PISA 
2015. Attainment 8 incentivises schools to encourage all pupils to 
take eight approved high-value subject qualifications, including 
mathematics, English and sciences. The intended benefit for pupils 
is to strive for good grades in fewer subjects, and for schools to 
place more emphasis on subjects such as mathematics 
(Department for Education, 2014). Progress 8 is a value added 
measure which compares Attainment 8 performance with 
performance at the end of primary school and is designed to look at 
performance across the ability distribution rather than focussing 
attention on the C/D borderline. 

Parents of PISA 2015 pupils had access to the support offered 
by Sure Start children’s centres. 

In 1999, the Sure Start programme was introduced around the UK. 
This Government programme aimed to help families from 
disadvantaged areas, by supporting children’s learning skills, health 
and well-being, and social and emotional development. PISA 2015 
pupils would have been one of the first year groups to experience 
these programmes, which worked with families whose children were 
not yet old enough for schools. 


A range of policies were introduced targeted specifically at 
disadvantage. 

Although only affecting a subset of the pupils participating in PISA 
2015, from 2003 onwards London Challenge was a substantial 
initiative intended to tackle low performance among schools in the 
capital. It followed the introduction of the first sponsored academies 
in 2000 and Teach First in 2002, which - whilst both initially small in 
scale -were explicitly focussed on schools serving large proportions 
of disadvantaged pupils. All three initiatives initially focussed on 
disadvantage and secondary schools, and so had plenty of time to 
have established by the time the PISA 201 5 pupils started 
secondary school. 

Mathematics for everyone 

In this report, we focus specifically on performance in mathematics 
to answer three questions. Firstly, how well is England doing at 
supporting the mathematics performance of disadvantaged 
pupils compared with other countries, and has this changed 
over time? 

When looking to minimise differences between disadvantaged 
pupils and their more advantaged peers, are there aspects of 
mathematics in which disadvantaged pupils are weaker and 
support could be targeted? 

And finally, what can we learn from pupils who beat their odds and 
do well despite their disadvantage? In what circumstances do 
disadvantaged pupils tend to perform better, and are there 
lessons that can be applied more widely? 
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4 What is disadvantage? 

Defining disadvantage 

Although the educational attainment of disadvantaged young 
people is a widely discussed policy and practice issue, detailed 
consideration of what we really mean by disadvantage is much 
rarer. Indeed, in much analysis and discussion, eligibility for free 
school meals is taken as definitive. The situation is, however, rather 
more complex. 

A theoretical framework that has been widely used in educational 
research for the past two decades is the intergenerational 
persistence model (Haveman & Wolfe, 1995). According to this 
model, children’s achievement is assumed to have two determinant 
factors: heredity and home investment. Heredity refers to innate 
traits and attributes that a child inherits from his/her parents, and 
home investment refers to time and a series of “goods” that a family 
needs to provide to ensure that the child will do well in school. 

“Goods” can be concrete (books, a school kit or a healthy packed 
lunch); services (such as paid tuition or joining an after school club); 
or intangible (e.g. providing the child with an extensive vocabulary 
or the confidence to speak in public and answer a teacher’s 
question). In particular, quality schooling can be considered both an 
essential and a premium “good”, and one which requires a 
considerable investment for many families. 

In the same way that each family or household has a certain level of 
income that allows its members to afford the goods and services 
necessary to their daily lives, families with school age children 


possess “educational capital” that allows them to provide a child 
with what’s necessary to do well at school. Similarly, we can think of 
poverty/disadvantage in the school context as the lack of sufficient 
educational capital to afford the necessary “goods”. 

The intergenerational persistence framework is illustrated below. It 
shows that “educational capital” is the combination of three different 
types of capital: economic, cultural and social: 

• Economic capital (a family’s income) is an important 
component of overall educational capital. The links 
between poverty and a child’s educational outcomes are 
universally acknowledged and a target of policy for many 
governments. However, it is only part of the picture: 
economic capital is not equivalent to educational capital. 

• Cultural capital refers to “non-financial social assets 
that promote social mobility beyond economic means” 
(Bordieu, 1986). In a school context, cultural capital 
includes parents’ education, occupation, knowledge, and 
the cultural ‘consumption’ and practices within the 
household. Indeed, parents’ education has been shown 
to be a more powerful predictor of children’s educational 
attainment than household income alone (Ganzach, 
2000; Sullivan & Brown, 2013; Sullivan et al., 2013). 

• Social capital is defined by the Organisation for 
Economic Co-operation and Development (OECD) as 
“networks together with shared norms, values and 
understandings that facilitate co-operation within or 
among groups” (OECD, 2001) and relates in the school 
context to how well family members can relate and 
interact with each other, the community, and schools. 
Families that experience domestic violence; migrant 
families that experience discrimination or are not familiar 
with the host-country’s social context; and single-parent 
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families can all be seen as disadvantaged in terms of 
social capital. 


Economic, cultural and social capital is interconnected, and 
generally the most disadvantaged households are deprived of all 
three. However, as we move up the educational 
advantage/disadvantage spectrum a more nuanced picture 
emerges. For instance, there are children in lower income 
households who are endowed with substantial resources in terms of 
social and cultural capital, and also children that although not living 
in poverty are nevertheless deprived socially or culturally. 

Figure 3.1 Intergenerational persistence of disadvantage 



Source: Adapted from Haveman & Wolfe, 1995 


Measures of disadvantage 

At the present the DfE’s main way of identifying disadvantaged 
children is those in receipt of free school meals (FSM) or who have 
been recipients at any given point in the previous six years (FSM6) 
(although additional support is also available for other groups, such 
as children who have been in care). The main advantage of these 
measures is that they are easy to compute and rely on a simple 
piece of information that is already available from schools and local 
authorities. However, as a measure of disadvantage FSM has three 
major shortcomings: 

1 . It groups all children into just two broad categories (FSM or 
not), not acknowledging that educational chances vary 
incrementally across the socio-economic spectrum. As a 
consequence, FSM often fails to identify children of families 
marginally above the eligibility threshold and of families that 
are just about managing but don’t qualify for benefits (so- 
called “JAMs”) as disadvantaged. 

2. It relies on income-level as a proxy for poverty, with anyone 
earning more than £16,190 per year being automatically 
disqualified. However, income level per se is not an 
adequate characterization of economic capital, since factors 
such as housing or childcare expenses, number of children 
in the household or even regional variations have to be 
taken into account. 


3. It disregards other important forms of disadvantage 
discussed above, particularly children whose lack of 
educational capital is mainly due to a shortage of social or 
cultural capitals. 


Is mathematics education in England working for everyone? NFER analysis of the PISA performance of disadvantaged pupils 8 







For PISA, the OECD has developed an alternative measure of 
educational capital, the index of economic, social and cultural status 
(ESCS). Children scoring below the OECD average for ESCS are 
classified as being disadvantaged. ESCS is calculated as a 
weighted average of three indices, generated from PISA surveys: 

• HISEI - the highest occupational status of a pupil’s 
parents. 

• ISCED (PARED) - the highest educational level of a 
pupil’s parents (converted into years of education). 

• HOMEPOS - an index of home possessions (measures 
the availability of 17 household items that act as 
measurements of family wealth and also the number of 
books at home). 

• By taking socio-cultural factors into account and 
providing a more sophisticated measure of economic 
capital ESCS addresses many of the weaknesses of 
FSM. Unsurprisingly, ESCS is therefore a better 
predictor of educational achievement than FSM 
(Knowles & Evans, 2012), and previous (unpublished) 
analysis of PISA 2009 and 2012 results by NFER 
statisticians has suggested it could explain up to twice 
the variation in achievement. 

ESCS also has the further advantage of being available for pupils 
from every country participating in PISA, and therefore offers an 
ideal lens through which to examine how well England is supporting 
the mathematics performance of disadvantaged pupils. 



What is ESCS? 


For the purposes of this report we are using the Economic, 
Social and Cultural Status (ESCS) index that’s estimated 
for every student who participates in PISA. ESCS is based 
on pupils’ responses to questions about their parents’ 
background and education, and possessions in their 
homes. The index is set to a mean of zero across OECD 
countries, with a standard deviation of one. More detailed 
information about how the ESCS index is calculated is 
provided in the PISA technical report. 
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5 How well is England 

supporting the mathematics 
performance of disadvantaged 
pupils? 

The gap in mathematics performance between the most and 
least disadvantaged pupils in England is equivalent to over 
three years of schooling. 

In common with other PISA countries, socio-economic status is 
associated with attainment in mathematics in England. The 
difference in achievement for the bottom ten per cent and the top 
ten per cent of pupils is 99 points, which is equivalent to over three 
years of schooling and close to the OECD average of 107 points. 
This is equivalent to the most advantaged pupils in England 
achieving at the average level of pupils in Hong Kong and the least 
advantaged pupils at the average level of pupils in Greece. 

Socio-economic status affects the mathematics achievement 
of all pupils 

Unlike the FSM measure, which only enables us to make 
comparisons between the performance of two groups of pupils 
(roughly 15 per cent of low income pupils and everyone else), 
ESCS allows us to make comparisons at every level of society. 
Figure 4.1 illustrates how for pupils at every point along the scale, 
slightly more ‘advantaged’ peers on average do better than them. 
Socio-economic status is therefore an important factor for all pupils 
- not just the least advantaged. If we are to fully de-couple life- 


chances from socio-economic status policy will therefore need to be 
wide-ranging, addressing every level of society, not just the lower 
rungs. 

Figure 4.1 Average mathematics score by ESCS decile in 
England 



Source: NFER analysis of OECD PISA 2015 
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Socio-economic status has a smaller impact amongst the most 
disadvantaged compared with the least advantaged. 

Despite these large differences across the range of socio-economic 
status, there is some evidence that socio-economic status has a 
smaller impact on mathematics performance for pupils with below 
average socio-economic status compared with those who have 
above average socio-economic status. This is evidenced from the 
smaller increases in score between the 1 st and 5 th deciles (25 
points) compared with the greater increase between the 5 th and 10 th 
deciles (72 points). 

This effect warrants further investigation, however one possible 
reason for it could be the impact of school funding linked to FSM 
eligibility (an important component of the school funding formula 
even before the introduction of the pupil premium) and - especially 
since 201 1 - the focus on providing additional support to this group. 

Across the UK, disadvantaged pupils participating in PISA 2012 
reported receiving slightly more contact time in mathematics 
courses than advantaged pupils (OECD, 2016a). This was only 
seen in a few other countries e.g. Germany, Sweden and 
Switzerland. 

Although socio-economic background is important, there are 
many other factors affecting performance. 

Whilst the difference between the average pupil from high and low 
socio-economic background is large, there is also a lot of variation 
in performance within these groups. The percentage of the 
explained variance in mathematics performance for England is only 
1 1 per cent (lower than the OECD average of 13 per cent). This 


indicates that factors other the ESCS have a greater impact in 
England than across the OECD. 

This could be a consequence of pupil-related factors such as other 
forms of disadvantage, linked for example to gender, race or 
regional disparity. 

It could also be a consequence of school-related factors. Research 
on the pupil premium in England undertaken by NFER (Macleod et 
al., 2015) found that although schools’ intake and circumstances 
are influential in predicting attainment of disadvantaged pupils, 
schools have meaningful scope to make a difference - through 
effective leadership and intervention. 

The research found that schools use a large number of strategies in 
order to improve the performance of disadvantaged pupils. Those 
considered most effective by schools were those which focused on 
teaching and learning, especially paired or small group additional 
teaching, improving feedback, and one-to-one tuition; all of which 
are supported by evidence of effectiveness in the Sutton Trust / 
Education Endowment Foundation’s Teaching and Learning Toolkit . 

Leaders in schools that were more successful in raising the 
attainment of disadvantaged pupils emphasised that there was no 
single intervention that had led to success. Rather, more successful 
schools appeared to be implementing their strategies in greater 
depth and with more attention to detail. More successful schools 
saw raising the attainment of disadvantaged pupils as part of their 
commitment to help all pupils achieve their full potential. They 
prioritised quality teaching for all, seeing attendance, behaviour and 
emotional support as necessary but not sufficient for academic 
success. 
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Differences in achievement are also apparent using TIMSS’ 
‘books at home’ measure of socio-economic background. 

TIMSS uses a ‘books at home’ measure to obtain pupils’ socio- 
economic status. The results from 2015 for England found there 
was an association between this measure and pupils’ average 
achievement in mathematics (Greany et al, 2016). Disadvantaged 
year 5 pupils (with 0-10 books at home) had an average 
mathematics achievement score 90 scale points lower than their 
advantaged peers (pupils with more than 200 books at home). For 
year 9, the scale point difference was 1 1 8. 

A similar effect was seen when using FSM as the measure of 
disadvantage. For both year 5 and 9 non-FSM pupils achieved a 
higher average achievement score in mathematics than their FSM 
peers, with a difference of 41 scale points and 42 scale points 
respectively. Whilst year 5 FSM pupils scored above the 
international mean in TIMSS 2015, year 9 FSM pupils scored below 
the international mean in mathematics. 
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6 How does England compare 
with other countries? 

There are two different ways to consider the impact of ESCS 
on mathematics scores across countries. 

PISA data allows us to make two types of comparisons by socio- 
economic status - absolute and relative: 

• An absolute ESCS comparison compares the 
achievement of two pupils in different countries with the 
same ESCS (for example, it compares mathematics 
scores for a pupil in England with an ESCS of 0.2 with a 
pupil in Estonia who also has an ESCS of 0.2). It is 
useful insofar that the two pupils should share similar 
backgrounds, and so in theory should face similar 
barriers to learning. However, due to overall differences 
in the distribution of ESCS between the two countries, 
the two pupils may find themselves in very different 
positions relative to their peers. 

• A relative ESCS comparison compares the 
achievement of two pupils who are at the same point in 
their country’s ESCS distribution (for example, it 
compares mathematics scores for a pupil from among 
the most deprived ten percent in England with a pupil 
from among the most deprived in Estonia). Although 
they may have different ESCS scores to one another, 
they both represent the most disadvantaged within their 
own societies. 

The type of comparison that is most appropriate depends on what 
we are most interested in: how pupils with a particular level of 
deprivation do compared with other countries, or how different 


countries manage deprivation and inequality within their country. 

For the purposes of this report we are particularly interested in the 
latter, given the current policy narrative of ‘a country that works for 
everyone’, and so our analysis focuses on relative ESCS 
comparisons. TIMSS, on the other hand, uses a scaled score to 
measure socioeconomic status to determine an absolute measure 
to use as a country comparison. Therefore caution should be taken 
when comparing disadvantaged pupils from TIMSS 2015 to PISA 
2015. 

On average, pupils in England have a higher socio economic-status 
(0.21) than pupils across the OECD (0). This means that when 
comparing pupils at the same point in their countries’ ESCS 
distribution (i.e. making relative comparisons), pupils in England will 
tend to be more advantaged in absolute terms. 

On average disadvantage has less of an impact on 
mathematics achievement in England than in many other 
countries. 

In order to make comparisons between multiple countries, Figure 
5.1 plots the strength of the relationship between pupil background 
and PISA mathematics score on the y-axis. However, it is also 
important to consider a country’s overall performance - if pupils 
perform poorly in general, the strength of the relationship between 
pupil background and performance may be small, but equally not 
something to be emulated. Therefore, the x-axis plots all 
participating countries by their average score in mathematics. 
Countries we would be most interested in exploring are those with 
above average PISA performance and with a smaller than average 
relationship between ESCS and PISA mathematics performance. 
These are countries in the top-right quadrant. 
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Figure 5.1 Strength of relationship of ESCS and performance 
in mathematics 
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This is illustrated in figure 5.2 which shows the mathematics 
performance of pupils in England compared with the OECD 
average by decile of socio-economic status. 

Figure 5.2 Average mathematics score by decile in England 
compared with the OECD average 



ESCS deciles 

Source: NFER analysis of OECD PISA 2015 


60 

Source: NFER analysis of OECD PISA 2015 

Compared with the OECD average, disadvantage in England has 
similar level of an impact on mathematics achievement, and 
average mathematics performance across all pupils is very similar. 


If we compare England’s performance with other parts of the UK, 
England has the least equitable outcomes by pupil background. 
This is illustrated in figure 5.3, which shows the mathematics 
performance of pupils in each country of the UK by decile of socio 
economic status (i.e. the chart compares equality within each 
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country, not pupils of similar socio-economic status across 
countries). The steeper the slope of mathematics score the greater 
the impact on average that an increase in ESCS has on 
mathematics score of pupils. 

Wales’ flatter slope indicates the lower impact of ESCS on 
performance compared with other parts of the UK, but the position 
of its line below the other parts of the UK reflects its poorer 
performance overall. The ideal, therefore is to maintain a flatter 
slope, but shift the line upwards with better performance at each 
decile. Scotland and Northern Ireland have similarly equitable 
outcomes by pupil background and overall performance to England. 

Figure 5.3 Average mathematics score by decile in 

England, Wales, Northern Ireland and Scotland 



ESCS deciles 

Source: NFER analysis of OECD PISA 2015 


The Netherlands and Estonia perform better than England, and 
Estonia is also more equitable. 

Figure 5.4 presents the relationship for England compared with two 
high performing European countries: the Netherlands and Estonia. 
Both countries perform better than England, but whereas in Estonia 
ESCS has a smaller impact on outcomes (indicated by a flatter 
slope) in the Netherlands the average impact of ESCS is greater. 

Figure 5.4 Average mathematics score by decile in England, 
Netherlands and Estonia 
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Canada is an example of a high performing predominantly 
English speaking country with more equitable outcomes. 

Looking further afield at other predominantly English speaking 
countries, Figure 5.5 shows that not only does Canada perform very 
well in PISA overall, the educational outcomes of pupils in Canada 
are more equitable when pupil background is taken into 
consideration. This contrasts with the outcomes in Ireland and New 
Zealand which are more similar to England. 

Figure 5.5 Average mathematics score by decile in England, 
Ireland, New Zealand & Canada 



Source: NFER analysis of OECD PISA 2015 


The high performing jurisdiction of Macao is the most 
successful PISA participant at limiting the link between pupils’ 
background and performance. 

The performance of pupils in England is often compared with that of 
high performing South East Asian participants. But how much of an 
impact does ESCS have on pupils’ scores? Macao is particularly 
successful at limiting the link between ESCS and points score, but 
is a small city state in very different circumstances to England, and 
so comparisons should be treated with caution. In contrast, ESCS 
has a stronger impact on performance in Singapore and Korea than 
in England. 

Figure 5.6 Average mathematics score by decile in England, 

Singapore, Macao and Korea 



ESCS deciles 


Source: NFER analysis of OECD PISA 2015 
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7 How does this compare with 
previous years? 

The relationship between disadvantage and performance of 
pupils in England is very similar to the previous round of PISA. 

Figure 6.1 compares the performance of pupils in England by decile 
ESCS in PISA 2015 (solid line) and PISA 2012 (dashed line). The 
shaded area around the PISA 2015 line shows the error, that is the 
range that we can be 95% confident the true score for that decile 
falls within. The PISA 2012 score at each decile falls within the error 
range at each decile and therefore the impact of background on 
pupil performance relative to other OECD countries has neither 
increased nor decreased in the last three years. 

In order to consider how the variability of outcomes is changing 
over time among pupils of given ESCS, we can look at the 
percentage of pupils who are resilient in each PISA cycle. 

Resilience is explored further in Chapter 9, and is defined as pupils 
with high levels of disadvantage who nevertheless achieve highly. 
There is some evidence of slight improvement over time, with 
around 23 per cent of disadvantaged pupils classified as resilient in 
2006 and 2009, increasing to 26 and 25 per cent in 2012 and 2015 
respectively. 


Performance of pupils at each level of disadvantage in England 
has remained static since 2012 whilst performance of other 
OECD countries has fallen, on average. 

Whilst it is disappointing that performance of pupils in England at 
each level of ESCS is unchanged since 2012, the performance of 
pupils in other OECD countries has fallen, on average. This is 
illustrated in figure 6.2 where the 2015 OECD average (solid pink 
line) is lower than the 2012 OECD average (dashed pink line). 

Figure 6.1 Average mathematics score by decile of pupils 
participating in PISA 2012 and 2015 in England 



Source: NFER analysis of OECD PISA 2012, 2015 
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Figure 6.2 Average mathematics score by decile of pupils 
participating in PISA 2012 and 2015 in the OECD 



Source: NFER analysis of OECD PISA 2012, 2015 


The relationship between disadvantage and achievement is 
very difficult to break. 

The relationship between ESCS and average mathematics score in 
England has been stable since 2006. Although the average impact 
of a unit change in ESCS has reduced slightly since 2006 
(illustrated in Figure 6.3), this change is not statistically significant. 

When the same comparisons are made across all participating 
countries, only nine countries have significantly reduced the 
relationship between ESCS and attainment in mathematics 


between any rounds of PISA since 2006. In many of these cases 
changes were just one-off, and in other cases they were reversed. 
The only two countries to have seen a sustained reduction in the 
impact of disadvantage over successive rounds of PISA are the 
USA and Mexico. However, neither of these countries performs well 
overall, and are therefore are not countries we would necessarily 
seek to emulate. 


Figure 6.3 Average ESCS of pupils in England between PISA 
2006 and 2015 

50 


§ 4 5 



2006 2009 2012 2015 

Source: NFER analysis of OECD PISA 2006, 2009, 2012, 2015 
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8 Are there areas of 
mathematics at which 
disadvantaged pupils are 
particularly weak? 

The performance gap for disadvantaged pupils applies to all 
areas of mathematics. 

PISA 201 2 had a focus on mathematics and allows for further 
analysis to build a deeper understanding of pupil performance in 
different areas of mathematics (equivalent analysis focussing on 
science would be possible using PISA 2015 data). 

Figure 7.1 shows the average performance of disadvantaged pupils 
overall in mathematics and for each PISA mathematics content and 
process area, compared with their advantaged peers (blue dots) 
and the difference between the two groups (green bars). The two 
groups refer to pupils who are in the bottom and top 25 per cent of 
the ESCS distribution respectively. 

The gap between these two groups of pupils is very similar for each 
area, showing that disadvantaged pupils do not disproportionately 
under perform in any component compared with their advantaged 
peers. Disadvantaged pupils do perform significantly worse in 
Space and Shape compared with their overall mathematics 
achievement; however this is the case for advantaged pupils as 
well. 


There seems to be no pattern in the magnitude of differences 
between content and processes. The largest difference was 
observed in the formulate process. 

When compared with performance of other countries, England 
ranks in a similar position across content and process areas of 
mathematics. This further supports the notion that disadvantaged 
pupils in England do not show particular areas of strength or 
weakness. 


Mathematical literacy in PISA 


PISA assesses mathematics across four content areas 
(Change & Relationship, Quantity, Space & Shape, 
Uncertainty & Data) and three processes (Employ, 
Formulate, Interpret). This provides the opportunity to 
indentify how disadvantaged pupils are performing at 
different types of mathematical problems, and whether 
there are areas of particular weakness. 


Information regarding domains is featured in PISA 2012 
Assessment and Analytical Framework . 
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Figure 7.1 Score across domains for disadvantaged pupils and 
their advantaged peers 



Source: NFER analysis of OECD PISA 2012 
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9 In what circumstances do 
disadvantaged pupils tend to 
perform better? 

As discussed in Chapter 5, there is greater variability in outcomes 
for pupils with a given ESCS score in England compared with the 
OECD. There are, therefore, a higher proportion of pupils who “beat 
the odds” by achieving more than would be expected given their 
socio-economic status (and also more who do even worse than 
expected). This section explores whether there are pupil 
characteristics which are associated with better performance, in 
spite of socio-economic disadvantage - described in the remainder 
of this section as “resilient pupils”. 

f ; ~ t 

What is resilience? 

A resilient pupil is a pupil who outperforms her or his peers 
sharing the same socio-economic background. Resilient 
pupils are those who fall into the bottom quarter of their 
country’s socio-economic background distribution and the 
top quarter of their country’s performance distribution. 

V J 


Do resilient pupils have particular 
strengths? 

Resilient pupils tend to perform highly in mathematics as a whole 
rather than in specific domains. Figures 8.1 and 8.2 show that 58 
per cent and 60 per cent of resilient pupils are resilient in all 
mathematical content areas and all mathematical processes, 
respectively. In general, high performing, low ESCS pupils show an 
aptitude towards mathematics overall. 

Figure 8.1 Percentage of resilient pupils by number of 
individual domains for which they are resilient 



0 12 3 4 

Number of content domains 


Source: NFER analysis of OECD PISA 2012 

A small proportion of pupils (8 per cent) were resilient in none of the 
individual content or process areas, but were resilient overall. 

These pupils were consistent performers who, whilst not achieving 
in the top 25 per cent for any one area of mathematcics, do appear 
in the top 25 per cent for overall mathematics score. 
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Figure 8.2 Percentage of resilient pupils by number of 
mathematical processes for which they are resilient 


60 % 



Number of process domains 
Source: NFER analysis of OECD PISA 2012 

What are the characteristics of resilient 
pupils in England? 

To identify factors associated with resilience we undertook a 
regression analysis. This predicted the likelihood of a 
disadvantaged pupil being classified as resilient given their 
individual characteristics. It is important to note when interpreting 
these findings that they demonstrate an association, but do not 
prove causality. Indeed, it is possible that the factors identified are a 
consequence rather than a cause of better mathematics 
achievement. Nevertheless, this analysis serves to highlight areas 
for future attention by policymakers, practitioners and researchers. 

Figure 8.3 identifies factors which are significantly associated with 
resilience. The beta values give the relative magnitude of the 


effects and, therefore, allow factors to be compared. The higher the 
degree of significance, identified by the number of stars, the less 
likely the observed result would occur due to chance. Three factors 
in the regression were found to have a significant positive 
association with the likelihood of a disadvantaged child being 
resilient (highlighted in green), and four were found to have a 
significant negative association (highlighted in red). 

The sections below provide further details of the characteristics of 
pupils in England who show resilience in mathematics. 


Figure 8.3 Characteristics significantly associated with 
resilience 


Variable 

Beta & significance 

Self Confidence in Mathematics 

0.26*** 

Second Generation Immigrant 

-0.20* 

Belonging to School 

-0.18*** 

Truancy (late to school) 

-0.18** 

Exposure to Pure Mathematics 

0.15** 

Mathematics Intention 

-0.10* 

Age 

0.09** 


*(p < 0.05), **(p < 0.01), ***(p < 0.001) 

Source: NFER analysis of OECD PISA 2012 
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What are the characteristics of a resilient pupil? 

A resilient pupil is likely to be old for their year (autumn 
born), feel little belonging to their school, have had 
exposure to pure mathematics, have little intention of 
further study or work with mathematics, is confident in 
mathematics, arrives on time to school, and is not a second 
generation immigrant. 


Self confidence 

Relative to the other variables, self confidence in mathematics had 
the strongest association with resilience, and was found to be highly 
significant. Disadvantaged pupils who are more confident in 
mathematics were more likely to be high achievers. 

This is consistent with OECD research (2011, 2016a), but as 
already noted, this does not necessarily indicate causality. For 
example, whilst a pupil who is more confident in mathematics may 
go on to achieve better marks, the questions regarding self 
confidence in the PISA survey lend themselves to more positive 
answers from high achieving children. Nevertheless, self confidence 
is important for children to attempt a greater number and difficulty of 
questions which facilitates higher achievement. 

Second generation immigrant 

Disadvantaged pupils who are second generation immigrants are 
less likely to be resilient compared with native British counterparts. 
Compared with other factors, it had the greatest negative effect 


observed. However, the broad category of second generation 
immigrant used here may not be very informative due to it grouping 
together culturally and historically distinct ethnic groups. 

This is a surprising result, given that lower socio-economic status 
white British pupils are consistently one of the lowest performing 
groups in schools and have become a focus of policy (The 
Education Committee, 2014). At age 16, amongst lower socio- 
economic status children, all ethnic minority groups significantly 
outperform white British pupils, apart from black Caribbean boys 
(Strand, 2014). 

The reduction in likelihood is unlikely to be due to English being an 
additional language (EAL), since no similar effect is seen for first 
generation immigrant children (although first generation immigrants 
constituted a very small number in the sample, so even if there was 
an effect this would be hard to detect). Furthermore, several ethnic 
groups (likely to comprise 1 st , 2 nd and subsequent immigrant 
generations), have smaller socio-economic status attainment gaps 
in GCSE results when considered using FSM entitlement (Strand 
2014). 

How then can this be explained? FSM and ESCS are two different 
measures of disadvantage so comparisons need to be made with 
care. For example, as a more sophisticated measure of 
disadvantaged than FSM, ESCS may be accounting for the factors 
over and above income that normally contribute to other ethnic 
groups performing well. The same may be true of the other 
variables included in our regression model (such as self-confidence 
and sense of belonging) which could act as mediators for the more 
commonly observed positive association. In which case, the 
‘second generation immigrant’ effect observed here could be 
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picking up on other underlying factors associated with second 
generation immigrant status that are not usually detected by more 
straightforward analyses and which operate alongside these other 
observed factors. 

Sense of belonging 

A disadvantaged pupil’s sense of belonging to their school is 
negatively associated with resilience in PISA. The scale indices 
measured how much a pupil agreed with various statements about 
feelings of inclusion at school i.e. “I make friends easily at school”. 
Compared with other variables, it had a negative effect and was 
highly significant. This result is counter-intuitive, since one would 
expect a child who felt part of the school to be happier and more 
engaged, possibly resulting in higher performance. 

One possible explanation is that resilience is causing a feeling of 
‘otherness’ in the sense that they are less likely to match the ESCS 
standing of other high performing pupils and they outperform peers 
with similar levels of ESCS. It may also indicate that resilient pupils 
are independent and motivated learners who are succeeding ‘off 
their own back’ and feel little sentimental connection with their 
school (although other factors such as motivation for mathematics, 
interest and work ethic are not found to be significant). Another 
possibility is that high performing children and the aptitude they 
display may result in them not requiring as much support from staff 
as other children, reducing a sense of belonging. 


Truancy 

Truancy, measured by self-reported frequency of late arrival at 
school, is found to reduce the likelihood of a disadvantaged child 
being resilient. It showed a similar amount of influence as sense of 
belonging but a lower level of significance. This reflects OECD 
research which found the time spent in class by a lower ESCS child 
was a large predictor of resilience (OECD, 2011). 

To improve the performance of disadvantaged pupils and 
encourage resilience, some schools provide incentives to ensure 
pupils arrive on time. Although this may result in some 
improvements, it risks oversimplifying the issue. Consistently 
arriving late to school could indicate a low level of support and drive 
for academic success in the family home. Indeed, where parents of 
disadvantaged children are engaged in their learning, this can be an 
effective means to support improved achievement (Grayson, 2013). 

Pure mathematics 

Exposure to pure mathematics was found to have the second 
greatest positive effect, and third greatest overall, on the likelihood 
of resilience. OECD research (2016a) has found a similar effect 
internationally. 

The suggested explanation is that pure mathematics provides a firm 
basis of concepts and understanding upon which other 
mathematical tasks can be achieved. A great deal of attention has, 
therefore, been given to disadvantaged pupils’ lack of opportunity to 
learn and engage with pure mathematics; and suggestions that 
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engaging low performing, low ESCS pupils in pure mathematics will 
raise overall standards. 

However, care is needed with this approach. Greater exposure to 
pure mathematics for low achieving pupils may overwhelm them 
and have a negative effect on confidence. There may also be an 
issue of reverse causality whereby more able (and hence resilient) 
disadvantaged pupils are being streamed into higher ability classes 
which focus more on pure maths. 

Mathematical intentions 

Increased mathematical intentions, measured by whether a pupil 
intends to further study mathematics or work in a job requiring 
mathematics, had a weak but significantly negative effect on 
resilience. Again, this result seems counterintuitive, and is not 
reflected in related factors such as motivation and work ethic. 

It is important to note that we have not considered these pupils’ 
performance in other subjects, and it is possible that high 
performers in mathematics may also be achieving highly in other 
disciplines in which they have a greater interest in further study. 

Age 

The weakest of the positive effects was within year age 1 , with 
disadvantaged children born in the autumn more likely to be 
resilient than their peers who were born in the following summer. 
This is consistent with a range of other research, including 


1 All children were aged between 15 years and 3 months and 16 years and 
4 months when they participated in PISA 


Crawford, Dearden & Greaves (2013) who found age to be a 
significant predictor at GCSEs, A levels and beyond to higher 
education. It is nevertheless notable that the effect persists with the 
alternative outcome measure of PISA score, and even having 
controlled for a range of other factors. 

This finding suggests that policies targeted at supporting pupils who 
are younger within year could particularly help to tackle 
disadvantage. Further research into classroom practices that are 
effective in supporting younger pupils could also be undertaken so 
that schools are equipped to adopt these practices more widely. 

Gender 

Notable by its absence, gender was not found to be a significant 
predictor of resilience. This is despite boys achieving significantly 
higher than girls in PISA mathematics overall when ESCS is not 
taken into account (Wheater et al 2013). The absence of a gender 
effect could be due to the counteracting effect of underperformance 
among low socio-economic status white British boys (Sammons et 
al., 2015; Tackey et al., 2011). 

However, this is not to say white working class girls do not 
underperform; compared with other ethnic groups of similar socio- 
economic status they are also underachievers (Education 
Committee, 2014). In fact, the parliamentary report on the 
underachievement of white working class noted that there has been 
too greater emphasis placed gender (Education Committee, 2014). 
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9 Conclusions and 
recommendations 

Conclusions 

On average England is supporting its disadvantaged pupils no 
worse than other OECD countries. Nevertheless, the gap in 
performance between our most and least disadvantaged pupils is 
equivalent to over 3 years of schooling at age 15. Furthermore, it is 
an effect that is felt at every point across the socio-economic 
spectrum. The international evidence shows that this gap is very 
difficult to shift, but it is lower in other countries, notably including 
Wales, Northern Ireland and Scotland. 

Our analysis has shown that disadvantaged pupils in England do 
not have weaknesses in particular aspects of mathematics, but 
rather their performance is weaker across the board. The fact that 
disadvantage affects all areas of mathematics, and every point of 
society, suggests that any solutions intended to close the gap will 
need to be far-reaching (Whitehouse and Burdett, 2013). 

One area for attention is grouping pupils by ability, which the 
evidence suggests is detrimental to the learning of low attaining 
pupils in mathematics, and is a particularly to the detriment of 
disadvantaged pupils as they are more likely to be in the low 
attaining group (Higgins et al., 2013). Setting can in turn lead to low 
ability pupils being exposed to less rigorous mathematics and so 
are not given the opportunity to reach their full potential (Schmidt 
and Burroughs, 2015). 


This issue is currently highly relevant due to Government plans to 
expand selection by ability in England. Critics of grammar schools 
argue that selective education amplifies tracking, having detrimental 
consequences for pupils who did not meet the level of proficiency 
required. However, supporters of grammar schools assert that it 
removes the socio-economic attainment gap rewarding academic 
ability rather than the ability to access resources. Either way, it is 
clear that tracking within schools should also receive attention 
during current debate surrounding allocation of pupils between 
schools. 

Recommendations 

Pupils would benefit from more sophisticated measures of 
deprivation. 

The impact of social and economic background of a pupil on school 
performance can be seen at every level in society; for any pupil, an 
increase in ESCS on average is associated with an increase in 
maths achievement. Therefore the dichotomous measure of 
eligibility of free school meals, used to allocate funding and target 
intervention, is not granular enough to work for everyone. New 
measures should be explored, for example using additional data 
sources to supplement if not replace FSM. These should also 
address forms of disadvantage linked to cultural and social capital 
in addition to economic capital. Equipped with new measures, 
policy should then be re-oriented around this broader, more 
nuanced view of the reach and nature of disadvantage in affecting 
educational outcomes. 
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Summer-born pupils need a strategy to ensure that they are 
not left behind. 

Summer-born pupils are less likely to be resilient than their autumn- 
born counterparts. The comparative poor performance of summer- 
born children is a consistent finding throughout the research 
literature. 

There have been suggestions that parents of summer-born children 
should be able to choose the year of schooling they enter, but this 
approach is likely to be to the detriment of more disadvantaged 
pupils: parents unable to afford an additional year of childcare may 
choose to send their children to school early and those from more 
affluent backgrounds may be more likely to delay the first year of 
school. This could then exacerbate the problem as instead of a 
class of pupils with up to 12 months difference in age, the age 
range could increase to 15 months. In addition, this approach still 
leaves some children the youngest in the year; and therefore, more 
targeted support for pupils young for their year may make more 
sense. 

Alternative strategies should therefore be identified, subjected to 
evaluation, and where successful adopted more widely. 

Further research on the characteristics of resilient pupils is 
needed to understand why some pupils beat the odds. 

There is more we can learn from pupils who beat their odds to 
perform highly in mathematics - pupils identified as ‘resilient’. This 
report identifies characteristics associated with resilience, but there 
is more that we would like to learn about these pupils. Linking the 
PISA database to the national pupil database and other datasets, 


such as school workforce data, will give us further clues on resilient 
pupils. 

Further measures are needed to support schools in adopting 
evidence-informed strategies, including through EEF. 

The absence of particular areas of weakness in mathematics 
among disadvantaged pupils, with poorer attainment seen across 
the board, suggests that schools should adopt a wide range of 
strategies to combat poor performance. However, it will be 
important to focus on interventions with evidence of effectiveness, 
and for schools to share learning, for example through a growing 
role for Research Schools and Teaching School Alliances. 
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